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ABSTRACT 

The "Trouble' Shooting Checklist" (TSC) is a 
diagnostic and predictive instrument designed to aid educational 
cha:.ge agents, faculty, and administrators in estimating the effects 
of particular variables on an institution's potential for 
successfully adopting innovations. The TSC consists of 100 
descriptive statements that are broken down into seven scales that 
indicate particular strengths and weaknesses within a school. Scale 
one (School Based Staff) contains items that are descriptive of the 
personality and leadership styles of principals, teachers, and 
counselors. Scale two (Communication) describes communication 
activities throughout the entire school systeia. The third scale 
(Central Administration) focuses on relations between the central 
offices, school, and sphool boardv Scale four (Innovative Experience) 
describes a school's experience with innovations and attitudes toward 
innovation. School/Community Relations, scale five, attempts to tap 
information on such variables as the amount and sources of funding^ 
the degree of interest^and involvement of community groups in the 
school system, and attitird-ajs^^of the community towards the schopl. 
Scale six (Organizational Climat^r)^-dagcribes the work climate and 
organizational structure of both the scE&oi-^and the district, and 
scale seven describes student behavior, attitudes, and demographic 
characteristics. Each item is rated on a five point scale ranging 
from "very typical" to "very atypical." The instrument provides seven 
scale scores in addition to a total score. Appendixes contain a 
subjective rating sheet, a reference list, tables of data, and a copy 
of the checklist. (DMT) 



ERIC 



The "Trouble Shooting" Checklist (TSC) 

For School-Based Settings 
(Mani-al) 



U $ OEPA^TMENTOF HEALTH. 
EDUCATION 4WELFAKE 
NATIONAL INSTITUTE OF 
EDUCATION 

TM'S DOCUMENT HAS BEEN REPRO- 
iJUCEO EXACTLY AS RECEIVED FROM 
THE PERSON OR ORGANISATION ORIGIN. 
A'.rNG iT POINTS OP VIEW OR OPINIONS 
STATED 00 NOT NECESSARILY REPRE« 
SENT OFFICIAL NATIONAL INSTITUTE OF 
EDUCATION POSITION OR POLICY 

Brad A. Manning 



The Procedures for Adopting Educational Innovations/CBAM Project 
The Research and Development Center 
for Teacher Education 
The University of Texas 
Austin, Texas 



The work reported here has been conducted with the support of the National 
Institute of Education, Contract No. NIE-C-74-0087 . The opinions expressed 
herein, however, are those of the author, and no endorsement by the National 
Institute of Education is implied. 



2 



Preface 



The "Trouble Shooting" Checklist for School-Based Settings \s one of two 
instruments designed to measure an organization's potential for successfully 
adopting and implementing educational innovations. In particular, the instru- 
ment focuses on a school's communication patterns, innovative experience, 
SQhool-based staff, central administration, relations with the community, or- 
ganizational climate and students. Another .>form of the TSC focuses on similar 
characteristics of departments in higher-educational settings (The ''Trouble 
Shooting" Checklist for Higher-Educational Settings) . 

c 

Research in the area of innovation adoption has primarily focused on three 
major areas: the adoption-diffusion process; characteristics of innovations 
which make an innovation easily adopted; and, identifying characteristics of 
organizations in terms of "innovativeness ." This instrument primarily takes 
advantage of literature in the last category and attempts to tie it in with 
the real-world experience of change agents. By identifying institutions which 
are not in a sufficient state of readiness to adopt innovations, the TSC can 
save potentially wasted time, efforts and money. As the development of new 
educational products and processes is becoming increasingly centralized, the 
number of innovations ready for adoption is rapidly increasing. Many institu- 
tions are seeking grants to adopt and implement these innovations without being 
sufficiently prepared to use the materials as the developer intended. The TSC 
should be useful for both change agents and in-house personnel in identifying 
strengths and weaknesses of an organization in relation to the adoption of in- 
novations . 

The first form of the TSC (Manning, 1973) was innovation-specific and 
focused only on h-'*cher-educational institutions. Subsequently, there have been 
three experimental, innovation-free forms of both the higher-educational and 
school-based TSC's. 120 institutions have been, rated on the various experimental 
forms of the TSC. These experimental forms, have resulted in the present two 
final forms. Since these forms have not been used in experimental or field 
studies, the author invites others ro use the TSC in research and development 
activities. j 

The development of these instruments was funded through the National Ins — 
tute of Education, the Procedures for Adopting Educational Innovations /CBAM 
Project at the Research and Development Center for Teacher Education, The 
University of Texas, Austin, Texas. In particular, I want to thank Gene Hall, 
the project director, for his support, and Ron Fox, Archie George, Sue Loucks, 
Beulah Newlove, Eddie Parker, Bill Rutherford, and all of the individuals who 
participated in the data collection. I want to especially thank Donna Buntair 
who has not only contributed her skills and expertise throughout the entire 
developmental period, but has provided me with invaluable encouragement and 
support. 

Brad A. Manning 

June 1976 
Austin, Texas , 
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General Description of the TSC 




potential for successfully adopting innovations. Two final formr of the TSC 
are now existant: the TSC for school-based settings (K - 12th grades), and the 
TSC for higher-educational settings (university and college level). This manual 
focuses on the school-based TSC, while another manual is available for the 
higher-educational TSC (see Manning, 1976). 

The TSC is a diagnostic and predictive instrument designed to aid the user 
in estimating the effects of particular variable's on the adoption-diffusion 
process. That is, the TSC provides users with a ireans of systematically organ- 
izing descriptive information in a predictive way. Because the TSC is broken 
into seven scales, a profile emerges, indicating particular strengths and weak- 
nesses within a school (with respects to the adoption process). 



II. Communications: This scale focuses on communication variables which 
significantly affect a school's potential for successfully adopting 
an innovation* In particular, this scale is concerned with patterns 
of communication (both within the school and the entire school system), 
initiators of communication, and types and forms of communication 
(with respect to both formal and informal channels of communication)* 



Scales 



The TSC consists of 100 



Likert-type 



items which can be broken into the 





TIT. Innpvat;'ve. .e.xpexi-eji-C£i: This scale focuses on the school's experience 
with innovations and attitudes towards innovation* Focus is on both 



past attempts dt innovation and present plans for innovation. Partic- 
ular variables which are considered are: the degree to which a 
school has prepared itself for the adoption of innovations; the reasons 
for considering adoption of innovations; the extent to which the school 
has realistically assessed its needs; the consultant role, the district 
role, and the community role in relation to both past and present plans 
for adopting innovations, 

Centrcl auiii nistrati on ; This scale focuses on relations between the 
central offices, school, and school board, and identifies attitudes of 
the central offices and school board toward innovation, their roles in 
relation to the school, and their awareness of the school's particular 
problems and needs, 

?£l}?2-l:/?-9^J^^-'V^^^y. This scale focuses on such variables as 

the amount and sources of funding, the degree of interest and involve- 
ment of community groups in the school system, the socio-economic en- 
vironment, and attitudes of the community towards the school. 

VI. Oroa ni -a clonal clip.ate This scale focuses on the work climate and 
organizational structure of both th j school and the central district 
office. Some of the particular organizational variables which are 
considered include: how decisions are made; how goals are established; 
what task groups exist; how task groups function; how planning takes 
place; what resources are available; how resources are used; how the 
organizational hierarchy is defined both within the school and the 
school district; and, the degree of centralization within the school 
district . 

VII. Students; This scale focuses on student behavior, attitudes and demo- 
graphic characteristics. Particular considerations include: student 
behaviors in the classroom and the lunchroom, absenteeism, tardiness, 
discipline problems, minority relations, teacher/student rapport, and 
academic excellence. 



Theoretical Framework and Ori gin 

The "Trouble Shooting** Checklist was developed in conjunction with a proj- 
ect which has as its theoretical framework the Concerns-Based Adoption Model 
(Hall, 1974; Hall, Wallace & Dossett, 1973). In brief, the Concerns-Based 
Adoption Mode] (CBAM) focuses on an individual's Stages of , Concern about, and 
Levels of Use of, an innovation. The ordering of concerns and use are postu- 
lated to be progressively more sophisticated throughout the adoption-implementa- 
tion process. The relationship of the TSC to the Concerns-Based Adoption Model 
is based on the assumption that in order for stages of concern and level''^ of 
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use to develop progressively through the adoption-implementation process, an 
institution must meet certain conditions and be in an appropriate state of 
readiness. The TSC is designed to aid in predicting and diagnosing an insti- 
tution's state of readiness^ 



Uses and Procedures' of Administration 

Uses of the TSC . The TSC has several major purposes. First, the TSC in- 
tends to provide an overall norm-referenced, predictive score which estimates 
the likelihood of a school to successfully adopt and implement an innovation. 
Secondly, the TSC intends to provide a seven scale diagnostic profile which 
focuses on the strengths and weaknesses of a school's environment in relation 
to the adoption and implementation of innovations. 

Within the context of being both a predictive and diagnostic instrument, 
the TSC may be used by several population groups for several different purposes: 

1. Change agents (either internal or external) may use the TSC as a pre- 
dictive tool to identify schools which would be most likely to adopt 
an innovation successfully. Such use of the TSC could assist a change 
agent or funding agency in determining whether commitments of time and 
money would be worthwhile in particular school settings. 

2. Change agents (either internal or external) may use the TSC as a 
diagnostic tool to obtain information that would help in planning in- 
tervention strategies appropriate to a particular school. For example, 
if a school scored particularly low on one scale, a change agent could 
plan interventions which would strengthen the school's particular weak- 
ness (e.g., establish new communication networks, etc.). 

3. Administrators and teachers interested in change may use the TSC to 
identify problem areas within their school. If a school is considering 
the adoption of a new program, members could use the TSC to self-- 
evpluate the school and identify strengths and weaknesses which would 
affect the adoption process. 

A. Several members of a school may complete the TSC in order to identify 
differences in their perceptions of the school. Such evaluation could 
be catalytic to discussions of problem areas and differences of per- 
ception within the school* 

5. Educators may use the TSC as an instructional tool to teach change 

agents or students interested in change, to identify key organizational 
variables which effect the adoption process. 

8 
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6. Research organizations may use the TSC to identify highly innovative 
or noninnovative schools for pilot testing of new programs, in order 
to measure the effects of the program in varied settings* 

Limitations on use . Users should keep in mind that the TSC is a new in- 
strument. While the TSC does have reliability „ some indications of validity, 
a^d norms based on a limited sample, the TSC has not been used extensively in 
either experimental or field studies. Therefore, the developer cautions users 
not to rely solely on TSC scores for decision making. 

Administra tion of the TSC . The TSC is easily completed and hand scored 
(see scoring section). The respondent is simply required to respond to each 
descriptive statement- on a 1 - 5 scale (ranging from "very typical'' to "very 
atypi^cXr'). In addition, the respondent may use either an "NA" or "?" to indi- 
cate if an item is Not Applicable to the particular institution (NA) , or iJf 
s/he does not have the necessary information (?)'. If a respondent uses a "?" 
for a response, s/he should try to obtain the needed information before scoring 
the instrument. Time required to complete the instrument, after an individual 
has familiarized him/herself with an institution, is estimated to be 20 to 30 
minutes. Scoring is estimated to take 15 minutes. • 

Members of an organization should be able to' complete most items based on 
their immediate knowledge of the institution. External change agents should 
first study the instrument in order to become completely familiar with the in- 
formation required to complete the form, and then spend a few days meeting with 
and interviewing various members of the organization in order to obtain the 
necessary information. The author urges that interviews be conducted with a 
variety of people, both receptive and unreceptive, of varying influence in the 
organization, in order to obtain the most complete impression of the institution* 
In such interviews, students should not be neglected. They can be a valuable 
source of infontiation. 
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Technical Development of the TSC 

Ori gin of the TSC 

The school-based TSC was an outgrowth of the higher-educational TSC which 
is documented in other reports (Manning, 1973; 1974; 1975). The items for the 
school-based TSC were collected from two sources: an extensive literature 
search for information describing innovative and noninnovative schools; and, 
45-minute interviews with ten practicing school-based change agents. The in- 
formation collected from the literature search was organized in the form of a 
series of referenced paragraphs summarizing study findings. Items v/ere written 
using these paragraphs as an information base. In taped interviews, school- 
based change agents were asked questions about the school and school district 
environment. The specific questions generally corresponded with the major areas 
which emerged from the literature. The tapes were then partially transcribed 
for information which would be likely to yield items, and items were subsequent- 
ly written. These items described either innovative or noninnovative school 
characteristics. Based on these interviews and the literature search, an item 
pool of 500 descriptive statements was collected and sorted into seven major 
areas of information: information on the school-based staff; information on 
communications; information on previous innovative experience; information on 
the central administration; information on school/community relations; infor- 
mation on Lhe organizational climate; and, information on the students. 

These 500 items were assigned to scales (corresponding to the seven in- 
formation areas), and randomly divided into groups of AO items. Each member of 
the project staff was asked to critique a set of items, indicating items which 
needed to be modified or eliminated. After this first screening procedure, 200 
items remained. These 200 items were used to buiid the second experimental 
form^of the school-based TSC (the first experimental form was a mock-up used 
for a workshop). The second experimental form was then given to organizational 



development specialists who critiqued each item in detail. Based on these 
critiques, the items were again re-written and the third experimental form 
emerged. The third form of the school-based TSC consisted of 200 randomized 
items in a Likert-type format. Each item described either an innovative or a 
noninnovative school characteristic. 

Item Analysis 

u 

Approximately 41 school-based personnel were asked to anonymously complete 
the third experimental form of the school-based TSC, Only 30 TSC's were re- 
turned in time to be included in the data base for the item analysis study. 

The first analysis focused on the following question: do items assigned 
to one of the two groups of items (items describing innovative ^institutions and 
items describing noninnovative institutions), belong with their respective 
groups? Because all participants did not respond to all items, the analysis 
was conducted twice: in the first analysis, items which were left blank were 
given the neutral value of ''3*'' (blanks = 3); and, in the second analysis, items 
which which were left blank were omitted from the analysis (blanks = omitted). 
Results of both analyses are recorded in Table 1. These analyses consisted of 
correlations between each item and the total score for each group of items 
(items describing innovative institutions and items describing noninnovative 
institutions). The alphas for the two groups were as follows: items classified 
as innovative a = .94; items clas ified as noninnovative a = .86; and total a 
for innovative and noninnovative = ,90 (when blanks were omitted from the ana- 
lyses) . The respective alpha coefficients for the tv^o groups(when blanks = 3) 
were as follows: .95, .87, and .92. 

The second analysis focused on whether or not it^fS^^re internally con- 
sistent within each of Ihe seven scales. This analysis consisted of correlations 
between each item and each of the total scale scores (see Table 2). As evidenced 
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in the result?*, every scale except Scale 1 (school-based staff) demonstrated a 
certain degree of overlap. All but one item in Scale 1 had a higher correlation 
with its own scale than with any other scale. These results indicate that 
institutions which rate highly in one scale are likely to rate highl^^ other 
scales as v/ell. 

The final analysis, like the second analysis, focused on whether or not 
items were internally consistent within each scale .and within the entire instru- 
ment (see Table 3). This analysis resulted in correlations of each item with 
the total scoire of the instrument and with total scale scores. The alphas 
of the seven scales and total (blanks = omitted) are represented in the table 
below : 



Scale No. 


Items in 
Scale 


Alpha 


1 


30 


.57 


2 


29 


.80 


3 


30 


.83 


4 


30 


.73 


5 


27 


.69 


6 


30 


.65 


7 


24 


.77 


TOTAL 


200 


.94 



Item Select-ion 
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The analyses described above provided tl-e basis for the selection of the 

final 100 TSC items. The selection procedure required the, tdmpilation of a 

detailed summary of the anal;^se ■ containing each item folloWfed by correlations 

{ 



on each analysis. In such a way, the results of all analyses could be reviewed 
at once, in order to determine which items correlated highest across the ana- 
lyses. In addition, items were examined in terms of content. When several 
individuals failed to respond to a particular item, the item was examined to 
determine if it required information that was difficult to obtain, or whether 
the item required information which was easily obtainable, but unknown to the 
respondent. In addition, particularly in borderline cases, the content of items 
was also considered in order to insure that a full range of variables would be 
represented within each scale. 

The final 100 items were then re-analyzed for new alpha coefficients in 
order to determine the reliability coefficients for the final five scales and 
total scores. The alphas for the final 100 TS'c items are presented in the table 
below: 



Scale No. 


Items in 
Scale 


Alpha 


1 


13 


. 79 


2 


15 


.89 


3 


16 


.92 


4 


14 


.84 


5 


13 


.82 •• 


6 


15 


.87 


7 


14 


.89 


TOTAL 


100 


.97 
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Format and Scoring 



Format 



The TSC consists of 100 descriptive statements. These statements are 

randomly arranged, and caw be broken into the following seven scales: 

Scale I: School-Based Staff 

(Items in this scale describe personality and leadership 
styles of principals, teachers and counselors,) 

Scale II: Communication 

(Items in this scale describe communication activities through- 
our the entire school system,) 

Scale III: Innovative Experience 

(Items in this scale describe a school's experience with inno- 
vations and attitudes towards innovation.) 

Scale IV: Central Administration 

(Items in this scale focus on relations between the central 
offices, school and school' board. ) 

Scale V: School/Community Relations 

(Items in this scale attempt to tap information on such vari- 
ables as the amount and sources of funding, the degree of 
interest and involvement of community groups in the school 
system, and attitudes of the community towards the school.) 

Scale VI: Organizational Climate 

(Items in this scale describe the work climate and organiza- 
tional structure of both the school and the district.) 

Scale VII: Students < 

(Items in this scale describe student behavior^'at titudes , and 
demographic characteristics.) ^^'^^ ^ 

Each item is to be rated on a five point "scale ranging from "very typical" 
to "very atypical." A may be used when one doesn't have enough information 
to respond to an item, and the ey.^Hol "NA" may be used when a statement is not 
applicable to a particular school. The instrument provides seven scale scores 
in addition to a total score. 
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Scoring 

Scoring of the TSC can be done by hand and requires approximately 15 min- 
utes. As explained in detail below, all items which describe noninnovative 
organizational characteristics must be reverse keyed before the scores are 
summed. Scores are then added for each scale and for the total instrument. 
Those respondents who have chosen to use the symbols ''NA'* and should refer 
to the Score adjustment f ormula section which provides a formula for equaliz- 
ing the scores of TSC's in which these symbols were used, with the scores 
of TSC's in which these symbols were not used. 

Reverse key scoring . The item numbers listed below are reverse keyed, and 
should have their rating values adjusted in the following manner: ^ 

reverse 
item score 
response value 



5 = 1 

4 = 2 

3=3 (reverse keyed items rated 3 
2=4 should not be changed) 

1 = 5 



For example, 


if 


you have marked 


one of 


the 


following 


items 


a "1," 


it should be 


changed to a 




for scoring purposes; 


if you have marked 


one of 


the following 


items a "4," 


it 


should 


be changed to a 


II2M 


for scpring purposes. 


The following 


items should 


be 


reverse 


keyed: 
















1 


13 


25 


39 


,46 


59 


78 


92 






2 


14 


28 


41 


48 


61 


81 


93 






4 


18 


30 


42 


49 


65 


82 


100" 






8 


22 


32 


43 


55 


72 


85 








9 


23 


38 


45 


58 


73 


87 







ERIC 



15 



11 



Scales . In order to derive each scale score, add the ratings for the re- 



\ 



spective item numbers listed below (the symbols^^?^' and" *'NA" should be assigned 
0 score values) : 



Scale 


I: 


5 


Al 


68 


85 






33 


' 59 


73 








35 


66 


78 








37 


67 


80 




Scale 


II: 


7 


2A 


60 


87 






16 


29 


75 


91 






19 


53 


76 


98 






20 


56 


82 




Scale 


III: 


3 


25 


A7 


64 






11 


32 


5.0 


79 






17 


A2 


58 


84 






23 


A5 


62 


100 


Scale 


IV: 


A 


22 


61 


96 






9 


30 


81 


97 






14 


36 


89 








21 


A3 


90 




Scale 


V: 


1 


27 


AO 


93 






12 


31 


AA 






- 


13 


3A 


52 








26 


39 


70 




ocaie 


V i : 


0 


JO 


J J 


OO 






8 


A6 


63 


92 






10 


A8 


72 


99 






18 


51 


7A 




Scale 


VII: 


2 


5A 


71 


94 






15 


57 


77 


95 






28 


65 


83 








49 


.69 


88 





Deriving total scor e. After the item ratings have been changed ai de- 
scribed above, add up all ratings to the left of items for the total score. 
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Score adjustment formula . If you have chosen to use , the symbols or 
"NA," it is necessary to use the following score adjustment formula. The 



score 



adjustment formula equalizes the scores of TSC's in which these symbols have 
been used with the scores of TSC's in which these symbols have not been used. 
This formula assumes that the items receiving a numerical response are repre- 
sentative of the entire scale content. All items rated "NA'* or receive a 
^ score value in this formula. 



Number of items in scale 



Actual 
computed score 

for scale 

Number 
of items not marked 
with a or "NA*' 
on scale 



For example, in order to score Scale I, first reverse key items as explained 
above. After reverse keying the items, add up the total score (giving and 
"NA" a score value of 0). If an individual ha? three "?'s," two *'NA's,** four 
"I's," two and two "3's," the formula would be completed as follows: 



Actual 
computed score 

for scale 

Number 
of items not marked 
with a or "NA" 
on scale 



= 14 



— X 13 (Number of 

items 
8 in scale) 



14 



X 13 = 22.75 (score for 
Scale I) 
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Similarly, the scora adjustment formula for the total score is as follows: 



Actual computed 
score for entire TSC 

Number of Items not 
marked with a or 

"NA" on entire TSC 



Number of items in TSC 



For example, if an individual rates an institution using five "?'s," five 
"NA's,'* ten "I's," ten •'2»s/' forty '»3»s,*' twenty "4's," and ten "5's*' 
the formula would be completed as follows (after reverse keying the items) 

280 



90 



X 100 « 311.11 
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Indications of Validity 

Due to limited resources at this time, a f^H scale study of validity has 
not been possible. However, some indications of validity are available. During 
tie item analysis study, respondents were asked to complete a subjective rating 
form (see Appendix A) on the same school which was rated on the TSC. On this 
form, respondents were asked to indicate their subjective assessment of the 
school's potential for successfully adopting educational innovations. On a 
scale of 1 - 4 (1 indicating no potential for innovation, and 4 indicating 
excellent potential for innovation), respondents were asked to provide an over- 
all rating and seven scale ratings. Each respondent's subjective ratings were 
then correlated with his/her TSC scores. These relationships were analyzed 
using a mult itrait-multimethod matrix (Campbell & Fiske, 1959), and examined in 
terms of convergent and discriminant validity in order to explore the possibility 
of any indications of validity. 

In order to establish convergent validity, there must be a significant 

correlation between two different measures of the same trait. Discriminant 

validity requires that: 

The correlations between two different methods measuring the 
same trait exceed (a) the correlations obtained between that 
trait and any other trait not having method in common and (b) 
the correlations between different traits which happen to 
employ the same method. Variance among test scores can be 
due to method and/or trait factors. The 'multitrait-multi- 
method matrix presents all the inter-correlations which 
result when selected traits are measured by two or more meth- 
ods (Borich & Bauman, 1972, p. 1031). 

Because the subjective ratings were made by the same person who compiGted 

the instrument, this study does not qualify as a true validity study. However, 

since this comparison is the only data available upon which to base indications^ 

of validity, the data will be analyzed as in a validity study. ^ 

1 O 
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Examining the four quadrant multitrait-multimethod matrix for the correla- 
tions between TSC ratings and subjective ratings (see Table 4), there is 
evidence of convergent validity. As evidenced in Table 4, all of these correla- 
tions are significantly different from zero, with the exception of Scale VII* 
These correlations, starting with Scale I through the total scores, are as fol- 
lows: .781, .670, .773, ,640, .478, .690, .049, and .642. Such correlations 
are indicative of the existence of convergent validity. 

The first method of establishing discriminant validity is to determine if 
the values in the validity diagonal (see Table 4) are higher than the values in 
the cojresponding rows and columns of the adjacent correlation triangles 
(heterot rait-heteromethod triangles). For example, when .781 (subjective Scale 
I with TSC Scale I) is compared with the correlations across the quadrant (first 
row) and down the quadrant (first column), .781 is found to be higher than 13 
out of the 14 correlations. The rest of the correlations are as follows: 
.670 is higher than all 14 correlations; .773 is higher than 13 correlations; 
.640 is higher than 12 correlations; .478 is higher than seven correlations; 
.690 ^is higher than 13 correlations; .049 is higher than six correlations; 
and, .64 2 is higher than eight out of its 14 associated row and column correla- 
tions (of the six higher correlations, four were approximately the same). 
Using this/.procedure for determining the existence of discriminant validity, 
there are indications of discriminant validity only for Scales I, II, III, IV, 
and VI. 

A second procedure used to establish discriminant validity requires that 
the values in the validity diagonal are higher than the triangles in the first 
and fourth <iuadrants (heterotrait-monomethod triangles) . These triangles 
represent the common influence of the same method on the seven scales and 
total scores. In other words, this second criterion requires that the trait 
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variance should be larger than the method variance. As can be seen by examining 
the table, the first correlation, • 781 (reading down the validity diagonal) is 
higher than all but one of the 28 correlations in the heterotrait-monomethod 
triangle directly above the validity diagonal. This same coefficient* (.781) 
compared to the hetero tralL-monomethod triangle in the fourth quadrant (sub- 
jective rating correlated with subjective rating) is higher than 11 out of 28 
correlations. The remainder of the correlations are as follows: .670 is higher 
than 22 correlations in the first quadrant, and higher than one correlation in 
the fourth quadrant; .773 is higher than all but one of the correlations in the 
first quadrant and higher than 11 correlations in the fourth quadrant; .640 is 
higher than 21 correlations in the first quadrant, and lower than all 28 corre- 
lations in the fourth quadrant; .478 is higher than 10 correlations in the 
first quadrant and lower than all correlations in the fourth quadrant; .690 is 
higher t^an 23 correlations in the first quadrant triangle, and lower than all 
but four of the correlations in the fourth quadrant triangle; .049 is lower 
than all the correlations in both the first and fourth quadrant triangles; and 
.642 is higher than 21 out of 28 of the correlations in the first quadrant 
triangle, but lower than all of the correlations in the fourth quadrant triangle. 
Thus, using this method, there is not evidence of discriminant validity. How- 
ever, the high inter-correlations among the subjective with subjective scales 
may indicate a unitary factor (see the discussion section) . 

A third criterion for the establishment of discriminant validity requires 
that the same pattern appear in all the trait-method triangles discussed above. 
As illustrated below, the four heterotrait (both inonomethod and heteromethod) 
triangles do have similar patterns of correlations (high and low are, of course, 
relative to each triangle, since the focus is on pattern and not size of the 
correlations). The table below describes this pattern: 





16 
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2 


3 


4 


5 


6 


7 


1 
















2 


high 














3 


high 


high 












4 


med. 


high 


med. 










5 


med . 


med . 


med • 


hich 








6 


^ high 


high 


high . 


high 


med. 






1 

7 


low 


low 


low 


low 


low 


low 




total 


high 


high 


high 


high 


high 


high 


low 



Using this method of determining discriminant validity, it can be concluded that 
an indication of discriminant validity does exist. 

Discu ^ jsion of Validity 

In analyzing the results of this study, it should be emphasized that the 
above di.^cussion is not a validity study in the usual, sense. " Instead, it is a 
study of the relationship between clinical subjective ratings of an institution's 
innovative potential (with respect to seven areas of an institution's environ- 
ment plus an overall rating) and the corresponding TSC ratings. In each of the 
37 institutions included in the analysis, both sets of ratings were made by the 
same individual. A true validity study would have, of course, contained an 
independent set of subjective ratings of the same institution made by a dif- 
ferent group of judges. In other words, in this correlational analysis, the 
approach, of necessity, was limited to having the same raters use a different 
means to rate the same institution. In a true validity study, the instruments 
would not only be different, but the raters would be different. 

Only one method of determining discriminant validity indicates validity 
(see method 3). The data suggest that the subjective scales do not discriminate 
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in measuring all the factors of the TSC scales and instead, measure a single 
factor. The eight ratings may be more reflective of a global attitude than of 
specific judgments of different aspects of an organizational environment due to 
the brevity of the subjective ratings. The possibility of a global attitude 
is buttressed by the fact that all of the correlations in the subjective with 
subjective quadrant correlate highly (see the second method for determining 
discriminant validity). In some cases, specific subjective ratings correlated 
higher with other TSC scales than with their own scale (e.g., TSC Scale III 
correlated higher with several other subjective scales than with its own scale). 

In sum, the results of the analyses of subjective ratings with TSC scores 
are inconclusive. The similar patterns which emerged in the third method of 
determining discriminant validity provide the only indications of discriminant 
validity. However, the fact that similar patterns emerge, provides evidence 
against the hypothesis of a unitary factor. Results of other methods of 
establishing discriminant validity as well as the higher inter-correlations 
of the fourth quadrant, suggest the possibility of a unitary factor. A true 
validity study must be conducted to determine whether a unitary factor exists, 
or Vrhether the instrument demonstrates discriminant validity. The author 
invites others to complete such a study or contribute information towards such 
a study. 

Finally, the student sca3e correlations (Scale VII) should be interpreted 
with caution. Many of the respondents reported that they were insufficiently 
familiar with the students to be confident about their ratings on this ^cale. 

Content validity is evidenced in the development section of this manual, 
as well as in two other papers (Manning, 1974; 1975). The instrument is based 
on both research literature and change agent responses to questions focused on 
the information areas contained in this instrument. In addition, other professional 

! 



researchers on the sponsoring project have offered their suggestions and 
critiques throughout the developmental process. Finally, organizational devel- 
opment specialists contributed suggestions for revisions. 



TABLE 4 

CORRELATIONS BETWEEN TSC RATINGS AND SUBJECTIVE RATINGS 

= 37) 
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Norms 

The norros are based on a group of 51 individuals who would be likely users 
of the TSC. They all worked in school settings in roles of teachers, adminis- 
trators, or change agents, and had an interest in the development of an instru- 
ment predictive of an organization's change potential. They filled out the 
instrument on a school with which they were familiar, with the understanding 
that they would not have to identify themselves or the school which they rated. 
A table of Percentile Equivalents for school-based TSC scores is presented in 
Table 5. This table gives percentile ranks and their corresponding raw score 
values for all seven scales and total score. In addition, means, medians, 
standard deviations and standard errors of the means are presented for each of 
the seven scales and total score. 

Of course, norms based on an N of 51 have limited value, but they represent 
a beginning. The instrument appears to be internally consistent, and indications 
of validity have been presented. Further use of the instrument seems to be 
warranted. The developer invites any institutions using this instrument to 
share their data, so that more extensive norms can later be published. It is, 
of course, also hoped that groups of institutions will generate their own norms. 
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Table 5 
PERCENTILE EQUIVALENTS 
FOR SCHOOL BASED TSC SCORES 
(N - 51) 
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Scale Score' Values 



II 



III 



IV 



VI 



VII 



Total 



73.00 
70.00 
65.00 
64.00 
62.50 
61.00 
59.00 
58.50 
58.00 
57.00 
36.00 
55.50 
55.00 
54.00 
53.00 
52.50 
52.00 
51.50 
51.00 
50.75 
50.50 
50. 2S 
50.00 
49.80 
49.60 
49.40 
49.20 
49.00 
48.00 
47.50 
47.00 
46.75 
46.50 
/'^.25 
4 .00 
45.80 
45.60 
45.40 
45.20 
^5.00 
44.00 
43.00 
42.00 
41.00 
40.50 
40.00 
39.00 
37.00 
36.00 
35.00 
31.00 
30.00 



78.00 

77.00 

76.00 

75.00 

73.00 

71.00 

70.00 

69.66 

69.33 

69.00 

63.00 

60.00 

59.00 

58.50 

58.00 

57.00 

56.00 

55.34 

54.67 

54.00 

53.66 

53.33 

53.00 

52.00 

51.66 

51.33 

51.00 

50.75 

50.50 

50.25 

50.00 

49.50* 

49.00' 

48.00 

47.00 

46.00 

45.75 

45.50 

45.25 

45.00 

44.50 

44.00 

43.50 

43.00 

42.00 

40.00 

39.00 

36.00 

33.00 

31.00 

26.00 

25.00 



92.00 

89.50 

87.00 

83.00 

82.50 

82.00 

81.00 

78.50 

76.00 

74.00 

70.00 

69.50 

69.00 

68.00 

67.75 

67.50 

67.25 

67.00 

66.00 

65.00 

64.50 

64.00 

63.66 

63. 3i* 

63.00 

62.50 

62.00 

61.00 

60.50 

60.00 

59.00 

58.00 

57.00 

56.66 

56.33 

56.00 

55.34 

54.67' 

5/1.00 

53.00 

52.00 

51.50 

51.00 

50.00 

48.00 

47.00 

46.00 

45.50 

45.00 

43.00 

32.00 

31.00 



85.00 
8,4.00 
80.00 
79.00 
75.00 
73.00 
68.00 
66.50 
65,00 
64.00 
62.00 
61.66 
61.33 
61.00 
59.00 
58.50 
58.00 
57.66 
^7.33 
57.00 
56.00 
55.80 
55.60 
55.40 
55.20 
55.00 
54.50 
54.00 
53.85 
53.68 
53.51 
53.34 
53.17 
53.00 
52.00 
51.66 
51.33 
51.00 
50.80 
50.60 
50.40 
50.20 
50.00 
49.00 
48.50 
48.00 
47.34 
46.67 
46.00 
43.00 
34.00 
33.00 



73.00 

70.00 

67.00 

63.00 

62.00 

61.00 

59.00 

58/25 ^ 

57.50 

56.75 

56.00 

55.66 

55.33 

55.00 

54.00 

53.66 

53.33 

53.00 

52.00 

51.75 

51.50 

51.25 

51.00 

50.66 

50.33 

50.00 

49.75 

49.50 

49.25 

49.00 

48.66 

48.33 

48.00 

47.00 

46.66 

46.33 

46.00 

45.75 

45.50 

45.25 

45.00 

44.50 

44.00 

43.00 

41.00 

40.00 

39.50 

39.00 

38.50 

38.00 

26.00 

25.00 



88.00 

86.50 

85.00 

84.00 

78.00 

74.00 

72.00 

71.00 

70.00 

69.00 

65.00 

64V0O 

63.00 

62.50^ 

62.00 

61.00 

60.00 

59.85 

59.68 

59.51 

59.34 

59.17 

59.00 

58.00 

57.75 

57.50 

57.25 

57.00 

56.50 

56.00 

55.00 

54.50 

54.00 

53.50 

53.00 

52.75 

52.50 

52.25 

52.00 

51.50 

51.00 

50.00 

49.75 

49.50 

49.25 

49.no 

48.00 
47.66 
47.33 
47.00 
40.00 
39.00 



74.00 
72.00 
71.00 
69.00 
66.00 
63.00 
62.50 
62.00 
61.00 
60.50 
60.00 
58.00 
57.50 
57.00 
56.66 
56. i? 
56.00 
55.00 
54.00 
53.00 
52.66 
52.33 
52.00 
51.50 
51.00 
50.00 
49.00 
48.80 
48.60 
48.40 
48.20 
48.00 
47.00 
46.50 
46.00 
45.00 
44.00 
43.00 
42.50 
42.00 
41.66 
41.33 
41.00 
40.50 
40.00 
38.00 
37. So 
37.00 
36.00 
34.00 
33.50 
33.00 



559.00 
"551.00 
536.00 
519.00 
482.00 
475.00 
452.00 
446.00 
445.00 
427.00 
420.00 
416.00 
412.00 
408.00 
400.00 
399.00 
•^96. 00 
39:>.C0 
393.00 
392.00 
391.00 
386.00 
385.50 
385.00 
381.00 
378.34 
375.67 
373^00 
369.00 
368.00 
365.00 
361.00 
*357.0O 
355.00 
352.00 
351.00 
348.00 
345.00 
341.00 
337.00 
333.00 
332.00 
330.00 
329.00 
324.00 
321.00 
319.00 
317.00 
316.00 
305.00 
226.00 
225.00 



50.08 



53.45 



63.02 



57.45 



50.86 



59.67 



51.22 



335.78 



50.00 



52.00 



63.00 



55.00 50.00 58.00 



50.00 



377.00 



8.62 



12.24 



12.96 



10.16 



8.66 



10.86 



10.07 



65.10 



Std. Error 
of Mean 



1.21 



1.71 



1.81 



1.42 



1..21 



1.52 



1.41 



9.11 
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SUBJECTIVE RATING SHEET 
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Subjective Rating Sheet 

Please assign a rating of 1 - 4 for each of the following categories, with re- 
spect to the school's potential for adopting innovations: 

4 = excellent potential for innovation 
3 = good potential for innovation 
2 = limited potential for innovation 
1 = virtually no potential for innovation 



1. Overall rating of school- 
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2. School-based staff . This category focuses on leadership and person- 
ality styles of teachers, principals, and counselors in relation to 
school innovativeness. Particular considerations should include in- 
terpersonal and professional interaction patterns, staff attitudes, 
previous working experience, and demographic characteristics of the 
school-based staff. 

3. Communications . This category focuses on communication variables 
which significantly affect a school's potential for successfully 
adopting an , innovation. In particular, this category is concerned 
with patterns of communication (both within the school and the entire 
school system) , initiators of communication, and types and forms of 
communication (with respect to both formal and informal channels of 
communications) . 

4. Innovative experience : This category focuses on the school's experi- 
ence with innovations and attitudes towards innovation. Focus is on 
both past attempts at innovation and present plans for innovation. 
Particular variables which should be considered are: the degree to 
which a school has prepared itself for the adoption of innovation; 
the reasons for considering adoption of innovations; the extent to 
which the school has realistically assessed its needs; and the con- 
sultant role, the district role, and the community role in relation 
to both past and present plans for adopting innovations. 

5. Central administration : This category focuses on relations between 
the central offices, school, and school board, and identifies atti- 
tudes of the central offices and school board toward innovation^ 
their roles in relation to the school, and their awareness of the 
school's particular problems and needs. 

6. School /community relations : This category focuses on such variables 
as the amount and sources of funding, the degree of interest and 
involvement of community groups in the school system, the socio- 
economic environment, and attitudes of the community towards the 
school . 
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Organizational climate s This category focuses on the work climate 
and organizational structure of both the school and the central 
district office. Some of the particular organizational variables 
which should be considered include: how decisions are made; how 
goals are established; what task groups exist; how task groups func- 
tion; how planning takes place; what resources are available; how 
resources are used; how the organizational hierarchy is defined both 
within the school and the school district; and, the degree of central- 
ization within the school district. 

Students : This category focuses on student behavior, attitudes, and 
demographic characteristics. Particular considerations should include: 
student behaviors in the classroom and the lunqhroom; absenteeism; 
tardiness; discipline problems; minority relations; teacher/student 
rapport; and academic excellence. 
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Table 

Items classified with scores of own scale (innovative/noninnovative) 
and with total score of instruiaent 



I terns 


classified as innovative 


Items classified 


as noninnovative 


correlated with total 


score or 


correlated with 


total score of all 


all items classified 


as innovar.ive 


items classified 


as noninnovative 


1 1 em 
Number 


Blanlcs=3 


Blanks=Oini t ted 


Blanks=3 


Blanks=Omitted 


1 






>8l5 


.H8H9 


2 




• 


.^*976 


.3901 


3 


.5839 


.5llt7 






)♦ 






.5823 


.5757 


5 


.5305 


.U716 






6 


.6101 


.U090 






7 


.5725 


.5682 






8 






.51H6 


. ^229 


9 






.5611 


.h36l 


10 


.7261 


.6976 






11 


.5255 


.5111 






12 


.6338 


.6230 






13 






.U668 


.^*391 


Ih 






.3392 


.3505 


15 


.5117 


.5132 






16 


.71^8 


.7012 






IT 


.5637 


.5360 






IB 






.3502 


.1*320 


19 


.56116 


.U987 




20 




.5595 






21 


.3810 








22 








. ^+627 


23 






.35^*5 




2h 


.7212 . 


.7008 






25 








.V<o6 


26 


.5623 


.5928 






27 


.6ll»5 


.U766 






28 






.568T 


. ^760 


29 


.5l)t5 


.505^* 






30 






.U621 


.i*071 



-This table includes information only on the final 100 items selected. Because all 
participants did not respond to all item's, this analysis was conducted twice: in the 
first analysis, items which were left blank, were given the value of "3" (blanks = 3); 
and in the second analysis, items which were left blank were omitted (blanks = 
omitted) . Results of both analyses are recorded above. 
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Table 1 (cont.) 



Items classified as 


innovative 


Items classified 


as noninnovat ive 


correlated with total score of 


correlated with 


total score of all 


all 


items classified 


as innovative 


items classified 

0 


as noninnovat ive 


Item 

IN UIIID C I 


Blanks=3 


Blanks=Omitted 


Branks=3 


Blank<?=Omi t ted 




• PP 1 ^ 1 














.U070 


.14135 


33 


.6691 










.6586 , 








35 




6l3)i 








. ODD J 


. DUD J 






37 




R7QQ 

* p 1 






38 






.3817 


.14620 








. 31H2 


.14151 




. 0 j±U 








111 






. U»i23 


.14690 








.6609 


.5578 


}i -3 






.5181 


.5939 




^0)iC 

.03^5 


. bOlo 












.6812 


.5869 


Ub 






.363U 


.^♦959 


UT 


C )i )i c 














. 7inU 


.5855 








.3609 


.3385 




• PPy J 


c:n7^ 
- PU f D 






pi 


'3o)t 


c).7^ 
. p^* ( b 






CO 


. 5do5 


- 5592 






CO 

Po 


t^OI )i 


. P fl3 






C)i 
p4 


. Uoll 








c c 

55 






, 52UU 


.5W9 


PD 




* PVii** 






C7 




* jU 






po 






.5667 


.5832 


CQ 

py 






. 521*6 


.5092 


oO 


^)iOft 


.pToo 






Ol 






.3735 


.1*031 


62 


J*052 


• 6193 






63 


.H366 


. 1*117 






6li 


J1965 








65 






.5151* 


.5078 


66 


.H792 


• 5109 






67 


.6767 


.6872 






68 


.61U6 


.5136 






69 


.ii827 


• H72O 






70 


.6192 


.6078 
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Table 1 (cont.) 



Items classified as innovative 


Items classified 


as noninnovative 


correlated with total 




correlated with 


total score of all 


all 


items classified 


as innovative 


Items classified 


as noninnovative 


Item 
Number 


Blank<;=2 


oianKs— umi t tea 


Blanks=3 


Blanks=Omitted 


71 


.II85U 


.5000 






72 






.1+1+73 


.3903 


73 






.3737 


.1+292 


7U 


.1+083 


.1+31*6 






75 


.5175 


.1+1+88 






76 


.U5itl 


.1+727 






77 


.31+31 


.2523 






78 






.1+625 


.381+2 


79 


.6828 


.6150 






80 


.5976 


.'+991 






81 






. 392l» 


.3728 


82 






.1+271+ 


.3623 


83 


.6305 


.6196 






8U 


.681+7 


.61+71 






85 






.3662 


.1+119 


86 


.5965 


.5521 






87 






.1+566 


.1+865 


88 


.2U29 


.2201 






89 


.5395 


.3352 






90 


.6305 


.621+3 






91 


.UllU 


.3959 






92 






.5290 


.6388 


93 






.1+676 


.l»7ltl+ 


9U 


.63lt6 








95 


•.lt768 


.1+821 






96 


.U5lt5 


.1+610 






97 


.7087 


.721(1 






98 


.501+3 


.1+711 






99 


.561+1+ 


.5151+ 






100 






.6253 


.6612 
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Table 2* 

Each item correlated with total scale scores for seven scales" 



Scales 



i tern 


Scale 
















Ml 1TT>V\ O Y" 
Vi kAulU C L 


NT 1 1 TTl r> Q >■ 


1 


2 


3 


4 


5 


6 


7 


1 


5 


-.288 






. UjO 




. 069 


. uuo 


2 


7 


_ 277 






— . UU L 


• 095 


. 17'^ 


.565 


3 






SP7 


. uou 






. D f i 


.311 


k 




17h 




mi 




)i c: cr 




.523 


5 


1 


.595 

* y y y 














C 


6 


• y -^y 












. 102 


7 


2 


.1^28 


525 

• yt- y 




. *♦ J J7 






).c;t 


R 




. Cl^7 


332 


1|2S 


P71 




. PJ-O 




J) 


1* 


-.052 


352 


liP^ 




, u u 


. 


c:)i)i 


10 




.h86 


567 


. 620 






. p'to 


^1 )i 


11 


^ 


.251 


536 


617 






. pyp 


. <iO<i 


1.^ 


5 


.690 


.iil9 

. -T J. ;7 


ii33 








. r ( u 


13 


5 


-.126 


-.15l» 


071 


.108 


lOS 


• UOD 


1 fip 


11; 




.1*67 


.1H8 


< .338 


563 




SI 7 


hns 

. H u ^ 


15 


7 


.359 


.li7ii 


285 


.386 


?08 


P^^ 


SSli 


16 


2 


.562 


.756 


.692 


.508 


.ii28 


6p8 


.526 


17 


? 


.Pft6 


. S7Q 






. H ( 




. ±Dh 


18 


6 


.535 

• y ^y 


515 

• y ^ y 


.IU»3 




PI T 


S77 




IQ 


2 


.339 


.529 
' y^ y 


31^3 

. «J ^ J 


• ^yy 


.H7I4 




1 1 A 


20 


2 


505 

' y^ y 


.689 


^IP 


I18O 




. 


0*37 


PI 


li 




^?7 


p6t 




111 0 


0'3C 

.<^35 


. J2o 


c c 


]. 






. ^ P ^ 


• { jU 


.>79 


.576 


.231 




J> 


hi S 


. ?c:o 


. DO^t 




.i9o 


, ^73 


.291 


Ph 


O 

c 


. jyc. 








.351 


. 5'*o 


)i he 


25 




.263 


.3^*5 


52h 


327 


2S8 




\ • ->^^ 
J.3Q1 


2t'. 


5 


.297 


.536 


.520 


.50l» 


.665 


M5 


27 


5 


.1»20 


.1|21 


M9 


.582 


.508 




.359 


?8 


7 


_.012 


.275 


.360 


.315 


.252 


Mo 


.259 


29 




.532 


.676 


.212 


.388 


.157 


.350 


.06I1 


30 




.23^1 


.350 


.537 


.361 


.560 


.3l»3 


.079 


31 


5 


.601 


.361 


.1|02 


.608 


,1»28 


.608 


.208 


32 


3 


.17'» 


.2l»3 


M5 


.173 


.210 


.16I1 


.150 


33 


1 


.614 1» 


.638 


.500 


MB 


.36I1 


.!»80 


.230 




5 


.51*1 


.555 


.629 


.570 


.580 


.598 


.563 


*This table includes information 


only on the 


final 


100 items 


selected. 


Items 



which did not receive responses from ^f^l participants in the sample were analyz 
only on the basis of the responses which were received (i.e.. Blanks = Omitted) 
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Table 2 (cont.) 



Scales 



Item 
Number 


b caie 
Ml imhp T* 


1 


2 


3 


4 


5 


6 


* 

7 


35 


1 


.769 


.lt7li 


.551* 


.523 


.1*12 


.1*1*6 


.171 


36 


It 


.731 


.520 


.326 


.639 


.313 


.1*36 


.081. 


37 


1 


.72i* 


.1*57 


.522 


.1*91* 


.507 


.396 


.1.81 


38 


6 


.550 


.361t 


.5ltl 


.571* 


.389 


.612 


.367 


39 


5 


-.065 


.179 


.333 


.198 


a377 


.187 


.1*65 


l«0 


5 


.li2lt 


.578 


.1*50 


.61*2 


U80 


.616 


.292 


1.1 


1 


.01*3 


-.178 


.Oil* 


.005 


.011 


.186 


.158 


h'c 


3 


.192 


.232 


.51*8 


.319 


.207 


.1*1*0 


.31*5 


H3 


li 


.237- 


.391* 


.627 


.1*39 


.1*31 


.628 


.392 


Ml 


5 


.5nf^ 


. l»7l» 


.531* 


.633 


.625 


.510 


.389 


l«5 


3 


-.02lt 


.29^ 


.577 


.226 


.301* 


.1*57 


.266 


U6 


6 


.235 


.21*9 


.1*11 


.327 


.529 


.355 


.186 


l«7 


3 


.1(77 


.318 


.li5lt 


.335 


.1*81 


.329 


.223 


l»8 


6 


.082 


.1(76 


.515 


.276 


.231 


.577 


.325 


l»9 


7 


-.097 


.060 


.017 


.129 


.069 


.201 


.563 


50 


3 


.363 


.lt3lt 


.615 


.1*11 


.500 


.1*39 


.286 ■ 


51 


6 


.1>33 


.591* 


.31*6 


.1*62 


.219 


.1*1*7 


.170 


52 


5 


.lt09 


.312 


.1*81* 


.1*28 


.1*71 


.507 


.593 


53 


2 


.1)05 


.597 


.229 


.532 


.133 


.1*79 


.090 


5n 


T 


.313 


.336 


.1*1*1 


.261* 


.1*18 


.376 


.783 


55 


6 


.302 


.1*1*1 


.519 


.359 


.31*6 


.552 


.287 


5r. 


2 


.1(28 


.667 


.1*31* 


.1*55 


.232 


.527 


.1*26 


57 


7 


.230 


' .262 


.212 


.352 


.388 


.292 


.627 


58 


3 


.155 


.290 


.1*73 


.21*6 


.1*98 


.385 


.373 


59 




,062 


.532 


.1*31 


.233 


.293 


.385 


.061 


(.0 


? 


.1(19 


.736 


.61*8 


.1*93 


.571 


.589 


.358 


61 


)t 


.300 


.1*03 


.28: 


.577 


.353 


.1*37 


.128 


r-2 


3 


.336 


.592 


.695 


.1*05 


.1*86 


.667 


.1*86 




5 


.1*23 


292 


397 


.1*26 


3U9 


.1*1*0 


1*07 


6n 


3 


.296 


.1*81 


.1*90 


.321* 


.337 


.287 


.11*1 


65 


T 


.33U 


.11*9 


.370 


.276 


.330 


.1.35 


.692 


66 


1 


.557 


.1*08 


.1*52 


.262 


.058 


.1*09 


.192 


67 


1 


.633 


.61*1 


.61*1 


.516 


.372 


.635 


.251 


68 


1 


.667 


.1*20 


.571 


.396 


.1*35 


.521* 


.1*82 


69 


7 


.388 


.231 


.361* 


.188 


.351* 


.283 


.628 


70 


5 


.1(31 


.1*52 


.610 


.536 


'.762 


.571 


.1*18 
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Table 2 (cont, ) 



Scales 



Item 
Number 


Scale 
Number 


1 


2 


3 


k 


5 


6 


7 


71 


7 


.388 


. 220 


lion 




li?7 


?8q 

• C\jy 


•108 


72 


6 




- 0'^8 


PPP 


P7P 


1 fill 




• 


73 


1 


.011 


- 035 


0li5 


. u 


• 090 


ISO 






6 


. ♦ 


.282 




. 0 ^ j- 


97"? 
. 1 J 






75 


2 


23^* 

• C i T 


5i|l 


hPP 






111 n 


3lifi 


76 

1 ' 


2 


.illl 


337 


613 




3Up 


1*78 


pliP 




7 


.196 


3l*3 


ii6l 

. ^ W JL 




i|l|P 


316 


li'^O 


78 


1 


• V ( U 




21 


138 


• U ( 0 


p63 


?nft 

• JVJU 


70 


'\ 


• t ^ w 


.620 


73? 


Uqp 






li?P 


8o 


1 




.1*78 




. z>y(- 


1*88 


In 1 




81 




-.067 


-,092 


35S 

• jy y 


.010 


• U^ ( 


PI Q 


1 QQ 


82 




.239 


.I126 


17U 


^96 


.121 


1|1|3 


13P 


83 


7 




.1*60 


503 


SI? 


630 

• ^ Jy 


386 


566 


all. 


3 




. 513 


.651 


.523 


.587 


.587 


. 269 


85 


1 


.1+89 


.191 


.1*58 


.298 


.lt2lt 


*532 


1*70 


86 


6 


.381* 




. **97 




.1+92 

. ^y *- 


. 559 
. ✓ ✓ ✓ 


.358 


87 


2 


.379 


.588 


.387 


.696 


.^75 


.571* 


. 231 


88 


7 


.078 


.133 


.267 


.222 


.368 


397 


5U7 


89 


1( 


.556 


.357 


.238 


.605 


• 277 


inii 


177 


90 


li 


.635 


51+7 


iil7' 


631 


296 


576 


. 189 


91 


2 


.2lt6 


.630 


1+97 


U53 


.1*20 


.1*05 


. 121 


92 


6 


.I1U7 


.lt56 


.531 


.U87 


.369 


.680 


.169 


93 


5 


.132 


.359 


.1*8U 


.357 


.511 


.230 


.113 


9lt 


7 


.525 


.380 


.532 


.Ult6 


.l»5Jt 


.1*93 


.652 


95 


7 


.271 


.328 


.332 


.U78 


.1*31* 


.395 


.506 . 


96 


li 


.187 


.396 


.295 


.U12 


.337 


.239 


.252 


97 


1* 


.611* 


.681 


.521 


.576 


.317 


.666 


.120 


98 


2 


.I1O3 


.5^*9 


.3^*8 


.2ltO 


.058 


.31+1 


.281 


99 


6 


.lt96 


.669 


.361 


.381t 


.111* 


.1*88 


.386 


100 


3 


.223 


.262 


.582 


.338 


.30l* 


.502 


.381 
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Table 3* 



Each item 


correlated 


with total 


scare of 


instrument 


and total scale 


score 




It ems 


classified as Innovative 


Items classified 


as Noninnovatlve 




R(Scale) 


R(Total) 


R(Sca 


le) 


R(Total) 


Item 


















Number 


B lanks 


Blanks 














Blanks 


B lanks 


U J.ciltl\.o 


D lcini\.b 


Rl pnk<5 


D J.CL I II^O 




= 3 


= Omitted 


= 3 


Omitted 






= 3 


= Dmi ttpH 

VJIU-L L L CU 


1 










.4450 


.4174 






2 










5952 




^ ^fi2 




3 , 


.6i63 


.6884 


.4934 


. 6224 










4 










. JUU u 




. 5134 


6032 


5 


. 5533 


. 5948 


.4371 


. 4922 










6 


. 4161 


.6034 


. 3245 


. 5347 










7 


. 503A 


. 5249 


.4669 


.4975 










8 












SI 7 7 


. 5064 


.4395 


9 










.2956 






.4165 


10 


. 4724 


. 5480 


.6337 


. 7000 










11 


. 5897 


.6174 


.5677 


. 5891 










12 


. 4076 


.4423 


. 5016 


. 5458 










13 










. 1819 


.1947 


n6s^ 


OS 70 


14 












. JO J £• 


5237 




15 


. 5240 


..5539 


. 4455 


. 4594 










16 


. 72?^ 


.755C 


.6881 , 


. 7330 










17 


. 5554 


. 5354 


.4985 


. 5623 










18 










.5203 


.5774 




.5210 


19 


. 5439 


.5291 


.3955 


.4633 










20 

o 


. 6601 


.6890 


.5052 


.5224 










21 


. 1860 


.4850 


.3295 


.3999 










22 










• 6430 


.7304 


.4629 


.6152 


23 . 










.6093 


6843 


.4417 


.5667 


24 


. 5994 


.6378 


.5720 


.6216 










25 










.5835 


.5240 


.5699 


.4707 


26 


. 7249 


.6654 


.6373 


.6011 










27 


. 5376 


.5081 


.4457 


.5742 










28 










.3307 * 


.2595 


.3375 


.3398 


29 


. 6408 


.6762 


. 3759 


.4229 










30 










.5157 


.3612 


.4101 


.44 58 



*Tlus table includes information only on the final 100 Items selected. Because all 
participants did not respond to all items, this analysis was conducted twice: in the 
first analysis, items which were left blank were given the value of "3" (Blanks=3); 
and, in the second analysis, items which were left blank were omitted (Blanks=Omitte(j) .- 
Results of both analyses are recorded above. 
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Table 3 (cont.) 





Items 


classified as Innovati've 


Items 


classified 


as Nonir.novative 




R(Scale) 


R(Total) 


R(Scale) 


R(Total) 


Item 


















Number 


Blanks 


Blanks 


Blanks 


Blanks 


Blanks 


Blanks 


B lanks 


Blanks 




= 3 = 


Omitted 


= 3 


= Omitted 


= 3 


= Omitted 


= 3 


= Omitted 


Ji 


.4148 


.4276 


.4418 


.5632 










'JO 










.4328 


.4453 


.3060 


.2869 


JJ 


.5518 


.6444 


.4976 


.5933 










Q /. 
JH 


.5399 


.5803 


.6616 


.7130 












.7157 


.7693 


.5011 


.5928 










Jo 


.5420 


.5455 


.4632 


.5355 










J/ 


.6615 


.7244 


.5111 


.6260 










Q Q 

Jo 










.5297 


.6123 


.5338 


.6001 


J9 










.1778 


. 3773 


.2191 


.2986 


/ r\ 


.4327 


.4797 


.5454 


.6180 










/. 1 

^1 










.0384 


.0426 


.C331 


.0348 












.5812 


.5475 


.4762 


.4141 


4J 










.2865 


. 4394 


.4125 


.5664 




.15603 


.6253 


.5686 


• .6474 




















.6162 


. 5766 


.4414 


.3892 












.2861 


. 3554 


.2477 


.4069 


/ "7 

47 


.3839 


.4537 


.3811 


.4615 










/ Q 

4o 










.6138 


.5773 


.5341 


.4539 


49 










.548"/ 


. 5629 


.1480 


.1582 


50 


.5839 


.6154 


.4533 


.5477 










51 


.4130 


.4467 


.4776 


.4779 






- 




5Z 


.3800 


.4706 


.5100 


.5608 










5J 


.6520 


.5968 


.5147 


.4400 










54 


.7434 


. 7831 


.4905 


.5112 










55 










.5590 


.5525 


.4414 


.5043 


56 


.6453 


■ .6671 


.5600 


.5652 










57 


.5930 


.6267 


.3989 


.4060 










c o 










.4612 


. 4727 


.4136 


.4314 


59 










.5363 


. 5322 


.3767 


.3708 


60 


.7274 


.73.^4 


.6153 


.6848 










61 










.4309 


.5769 


.4113 


.^396 


62 


.7292 


.6954 


.6837 


.6591 










63 


.4384 


.4397 


.4339 


.4794 










64 


.4933 


.4896 


.3843 


.4283 










65 










.7012 


.6924 


.4404 


.4466 


66 


.5942 


.5568 


.4472 


.4190 










67 


.6269 


.6334 


.6531 


.6608 










68 


.6095 


.6674 


. 5460 


.6140 










69 


.6014 


.6280 


.3952 


.4213 










70 


.7226 


.7616 


.6270 


.6679 
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Table 3 (cont.) 





Items 


classified as Innovative 


Items 


classified 


as Noninnovative 




R(Scale) 


Rdotal) 


R(Scale) 


R(Total) 


iceiu 


















Number 


Blanks 


Blanks 
















"D 1 onlro 


12 1 <^ m 1j> <r> 


bianks 


Blanks 


Blanks 




= 3 


Omitted 


= 3 


VJul J. ^> UcU 


— Q 
*- J 


— umiLLea 


— '5 

- J 


= Omitted 


71 


. 5265 


.5978 


.A507 


. A53A 










72 










.3162 


. 290A 




. ^4 04 


73 










.0074 


on 0 




• \)£.y i. 


74 


.3292 


.3630 


.3781 


. 3655 










" 5 


.5523 


.5A13 


.A250 


.A783 










76 


.AA71 


.3366 


.51A5 


.A76A * 










77 


. A320 


.A30A 


. 3931 


.445-7 










78 










. J. 0\j£. 


. U / JO 


9 AAA 


* loyo 


79 


.67A3 


.7331 


.6297 


. 7108 










80 


.3953 


.A1A2 


.A095 


. 5229 










81 










0000 


noQQ 

. yjyjyy 


. JL 7 


1 9Q1 


82 














. 4 loo 


. J44l 


83 


. 5292 


. 5663 


* J y J J 


. D J D 4 










84 


.6190 


.6512 


5795 












85 
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APPENDIX D 

THE '•trouble shooting" CHECKLIST (TSC) 
FOR SCHOOL-BASED SETTINGS 
(Instrument) 
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The "Trouble Shooting** Checklist (TSC) 
For School-Based Settings 
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Introduction and Instructions 

The TSC consists of 100 Likert-type items describing school characteristics. 
These 100 items are randomly arranged and can be broken into seven scales 

using on particular organizational variables which affect the adoption- 
diffusion process. The history of the development of the instrument is included 
in the manual, as well as information on uses of the TSC, scale titles, numbers 
of items in each scale, and scoring instructions. 

The respondent" is asked to rate whether or not an item is descriptive 
(typical/atypical) of a particular school. Although there are descriptive 
statements about individuals outside of the school (such as the superintendent) , 
the statements, nevertheless, focus on such a person in relation to the school. 
The school being rated should always be the point of reference. 

The terra change agent is used throughout the instrument. This term is 
used broadly, and includes both external change agents (individuals brought in 
from outside of the school system for the purpose of facilitating change) , and 
internal change agents (permanent members of the organization who are respon- 
sible for facilitating change). The role of the change agent may range from 
assisting a school in problem-solving by giving of their own professional 
expertise, to providing contacts with all available resources, to actually 
taking part in decisions to adopt innovations and aiding in the implementation 
of adopted innovations. It is also assumed that the school you are rating is at 
least considering the adoption of one or more innovations (as reflected in 
items referring to "innovations"). Finally, the term c urriculum specialist 
refers to any person who is responsible for the planning and development of the 
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curricurum. This person could be a permanent member of a school, or may be 

available through the central district offices for all schools in a district. 

Such a person may not be titled "curriculum specialist/* however. The TSC 

should be completed in terms of the person who fulfills the role of curriculum 

specialist. If no one fulfills this role, simply mark ''NA'' (Not Applicable) in 

blanks next to items referring to curriculum specialists. 

In order to complete the instrument, merely rate on a 1 - 5 scale (as 

indicated below), how closely each item describes the department you are rating, 

and record your ratings on the line directly to the left of the item. 

5 = very typical 

4 = somewhat typical 

3 = neither typical nor atypical 

2 = somewhat atypical 

1 = very atypical 

In addition, you may use one of the following two ratings: 

(1) if the item is applicable to the school you are rating, but you do 
not know the information, use the symbol "?." 

(2) if the item is not applicable to the particular school you are rating, 
use the symbol "NA." 
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The "Trouble Shooting*' Checklist (TSC) 
for School-Bas^d Settings 



Please rate oa a 1 - 5 scale (as indicated below) how closely each item describes 
the school you are rating, or use one of the alternative symbols; 



5 




very typical 


4 




somewhat typical 


3 




neither typical nor atypical 


2 




somewhat atypical 


1 




very atypical 


NA 




not applicable 


9 




no information at this time 



1. There is much concern from the state legislature over how the innova- 
tion will affect the curriculum. 

2. There are many discipline problems at this school. 

3. This school is considering innovations that contain easily alterable 
materials which can meet the demands of varied teaching situations. 

4. The central offices wait until there is a public outcry before they 
inform the school board of problems in the schools. 

5. Non^supporters of innovation work on committees and/or travel to school 
with supporters of innovation. ^ 

6. The board members are highly concerned about faculty-administrative^ 
board relations. 

7. Direct, two-way communication occurs between administrative staff and 
the teaching staff. 

8. The school district has an intricate bureaucratic system. 

9. The school places blame on the central office for most of its problems. 

10. Reasons for change are understood by members of this school. 

11. Analyses have been made concerning the effects of innovations on the 
entire school. 

12. It is part of the principal's job to maintain good relations with the 
community. 

13. The parents infrequently attend school events. 

14. The superintendent in this school system cannot withstand any criticism. 
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Please rate on a 1 - 5 scale (as indicated below) how closely each item describes 
"the school you are rating, or use one of the alternative symbols: 

5 = very typical 
A = somewhat typical 
3 = neither typical nor atypical 
2 = somewhat, atypical 
1 = very atypical 
NA = not applicable 
? = no information at this time 



15. The students respect individual differences among themselves, 



16. The teaches receive regular communications about what is happening in 

the school system at large. 

17. Change agents have been invited to return more than once for information 

on educational change processes. 

18. Teachers and principals do not jointly establish goals. 

19. Board members communicate often with the superintendent. 

20. The principal communicates well with the community. 

21. The board of education does not actively oppose innovations. 

22. The central office does not inform the school board about what is new 

in the field of education. 

23. The teachers at this school know very little about new educational 

practices. 

24. The principal often visits teachers' planning sessions. 

25. School personnel are pressured to change by the central school district 

office. 

26. The superintendent receives community opinion directly, rather than 

depending on central office staff to relay messages. 

27. This school system has effective representation in community politics. 

28. A larpe number of students at this school are receiving failing marks 

in rnelr coursework. 

29. The principal communicates face-to-face with teachers and/or staff, 

rather than by memo .-^r phone. 

30. The curriculum specialist does not have credibility with the teaching 

staff. 

31. Board members believe that the community supports innovation. 

of . 

ERIC 4V 



43 



Please rate on a 1 - 5 scale (as indicated below) how closely each item describes 

the school you are rating, or use one of the alternative symbols: 

5 = very typical 

A = somewhat typical 

3 = neither typical nor atypical 

2 = somewhat atypical 

1 = very atypical 

NA = not applicable 

? = no information at this time 



32. Innovation attempts up to this time have not been carried out success- 
fully on a day-to-day basis'. 



33. The principal attends meetings and conferences outside of the school 
district. 

3A. The parents are kept well-informed of school events, 

I 

35. The teachers at this school ask well-informed questions about instruc- 
tional procedures.' 

36. There are well-informed research and evaluation personnel at the central 
office . 

37. The teachers have developed some new classroom practices on their own. 

38. The atmosphere among most teachers is more competitive than cooperative. 

39. Applications for funding in this school district require specific infor- 
mation on procedures and/or evaluation. 

AO. This community has elected school board members who are interested in 
innovation. • 

Al. The counselor has i poor rapport with teachers, 

A2. Teachers are pressured from the central offices to implement innovations 
quickly . 

A3. The central administrative personnel favor innovations which do not 
alter the system^s overall structure. 

AA. The parents fee^l that they are able to give their opinions to the school 
administration. 

A5. Innovations have be^^i ir,posed externally in this school system without 
regard to specific local needs. 

46. There are no strong pressures for change outside of this particular 
school . 
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ease rate on a 1 - 5 scale (as indicated below) how closely each item describes 
e school you are rating, or use one of the alternative symbols: 

5 - very typical 
4 = somewhat typical 
3 = neither typical nor atypical 
2 - somewhat atypical 
1 = very atypical 
NA = not applicable 
? = no information at this time 

47. Members of this school have requested the opportunity to see, in 

operation, an innovation which is under consideration. 

48. There are several isolated subsystems in this school. 

49. The students are extremely rowdy in the lunch room. 

50. The school plans for implementation of innovations include systematic 

procedures for staff education. 

51. The process of decision-making is clearly defined. 

52. The parents have supported educational changes in the past.. 

53. The curriculum specialists have systematically collected information 

about the needs of the school through direct contact with teachers. 

54. The students feel that they are learning things which are relevant t;o 

them. 

55. Concrete activities, necessary for carrying out educational changes, 

have not yet been specified. 

56. The administrative staff communicates regularly with the teaching staff. 

57. Although these students have their ups and downs, they are not basically 

frustrated . 

58. This school would only be interested in making changes to avoid criti- 
cism from the school district central office. 

59. There are grave weaknesses in the channels and procedures for dissemi- 
nation. 

60. The priqcipal, teacher representative, or outside agenu meets with 

small groups to determine the groups* understanding of the innovation. 

61. All power lies in the central offices of the school district. 

62. The superintendent involves the teaching staff, especially during the 

decision-making phases of the curriculum change process. 
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Please rate on a 1 - 5 scale (as indicated below) how closely each item describes 

the school you are rating, or use one of the alternative symbols: 

5 = very typical 

4 = somewhat typical 

3 = neither typical nor atypical 

2 = somewhat atypical 

1 - very atypical 

NA = not applicable 

? = no information at this time 



63. Teachers have access to the kinds of resources which they feel they 
need. 

64. The person introducing the innovation has recognized authority in the 
school. 

65. The students don^t seem to be paying attention in the classroom. 

/ 

66. The teachers at this school interact . outside of school hours. 

67. Teachers at tthis school attend professional meetings outside of their 
school district. 

68. The teachers have some peer support system established to assist each 
other in their teaching responsibilities. 

69. The students work, well independently. 

70. This school system is sensitive to community opinions. 

71. To a large, degree , the students are self-directing. 

72. Groups of innovators and non-innovators have emerged in the form of 
in- and out-groups. 

73. The counselor serves the dual function of being an administrator and 
a counselor. 

74. The teachers are given sufficient time during the day to plan, elimi- 
nating the need to take most of their work home in the evenings. 

75. A school administrator initiates communications with the change agent. 

76. Teachers attend workshops teaching identification of problem areas in 
schools . 

77. The students feel that they have some control over their learning 
experiences. 

78. The school counselor has disciplinary responsibilities. 
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Please rate on a 1 - 5 scale (as indicated below) how closely each item describes 
the school you are rating, or use one of the alternative symbols: 

5 = very typical 
4 = somewhat typical 
3 = neither typical nor atypical 
2 = somewhat atypical 
1 = very atypical 
' NA = not applicable 

? = no information at this time 



79. Many types of instructional materials have been examined by members of 
this school system in order to determine what innovation would be best 
suited to their needs. 



80. A few of the teachers who have been with the school for a long time 
suppor t innovat ion . 

81. The central office is pushing the adoption of innovations for the bene- 
fit of federal money. 

82. The principal does not often ask for suggest-.ons from the faculty. 

83. Teachers and students have an informal rapport. 

84. Specific problems and needs have been identified by members of this 
school system. 

85. Most of the teachers leave school as soon as possible after the stu- 
dents are dismissed. 

86. The teachers at this school are encouraged to participate in summer 
planning sessions. 

87. The principal receives most communications from the central offices by 
way of memos. 

88. Students use some of their leisure time to do school related activities. 

89. The curriculum specialist evaluates in a constructive way. 

90. The central office provides individuals who are fostering research. 

91. The principal, teacher representatives, or outside agent meets with 
small groups of teachers in order to receive teachers* opinions. 

92. No channels exist in the organization for appealing decisions. 

93. Local agencies which control funds are vague about how the funds can 
be used. 

94. Students are viewed by the teachers as being participative rather than 
passive . 
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Please rate on a 1 - 5 scale (as indicated below) how cl9sely each item describes 
the school you are rating, or use one of the alternative symbols: 

5 = very typical 

4 = somewhat typical 

3 = neither typical nor atypical 

2 = somewhat atypical 

1 == very atypical 
NA not applicable 

? = no information at this ^ime 



95. At least one-third of the students at this school are children of 
professionals. 

_ 96. Decisions in the central offices are based on information contributed 
from all levels of the school system. 

97. The schoo] board helps to obtain funding for innovations which are 
initiated by the individual schools. 

^ 98. Teachers at this school can give their honest opinions to the principal 
with confidence . 

99. The principal encourages decision-making by consensus. 

100. Although the teachers have already been working with an innovation for 
some time now, they do not fully understand what the innovation is all 
about . 



